Papers with task-specific model
Coreference Resolution without Span Representations (2021.acl-short)
Copied to clipboard
| Challenge: | Pretraining has reduced many complex task-specific NLP models to simple lightweight layers. |
| Approach: | They propose a lightweight end-to-end coreference model that removes the dependency on span representations, handcrafted features, pruning heuristics, and more. |
| Outcome: | The proposed model performs competitively with the current standard model, while being simpler and more efficient. |
Reinforced Training Data Selection for Domain Adaptation (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to learn domains with massive data are not easy to implement and require a predefined threshold. |
| Approach: | They propose a framework that searches for training instances relevant to the target domain and learns better representations for them. |
| Outcome: | The proposed framework is effective in data selection and representation, but generalized to accommodate different NLP tasks. |
An Exploration of Three Lightly-supervised Representation Learning Approaches for Named Entity Classification (C18-1)
Copied to clipboard
| Challenge: | a recent study compares semi-supervised learning methods with bootstrapping methods . semi-semi-supervised methods reduce the amount of semantic drift introduced by iterative approaches . |
| Approach: | They propose to adapt three semi-supervised representation learning methods to an information extraction task . they show that all methods outperform state-of-the-art semi-representation learning methods . |
| Outcome: | The proposed methods outperform state-of-the-art semi-supervised methods on named entity classification task. |
Multi-View Cross-Lingual Structured Prediction with Minimum Supervision (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on cross-lingual transfer learning focuses on transferring knowledge from high-resource languages to low-resourced ones. |
| Approach: | They propose a multi-view framework that integrates multiple source models into an aggregated source view and transfers it to a target view based on a task-specific model. |
| Outcome: | The proposed framework improves on three structured prediction tasks on 16 datasets. |
UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on geometry problem solving treats calculation and proving as two specific tasks hindering a deep model to unify reasoning ability on multiple math tasks. |
| Approach: | They propose a large-scale Unified Geometry problem benchmark to unify geometry on multiple math tasks. |
| Outcome: | The proposed framework outperforms the existing model with 5.6% and 3.2% accuracies on calculation and proving problems. |
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language models (PLMs). |
| Approach: | They propose a progressive zero-shot dataset generation framework which leverages feedback from the task-specific model to guide the generation of new training data via in-context examples. |
| Outcome: | The proposed framework achieves on-par or superior performance with only 1% synthetic dataset size, when compared to baseline methods without in-context feedback. |
Dynamic Low-rank Estimation for Transformer-based Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | RankDyna is a matrix decomposition method that can be used to compress Transformer-based language models. |
| Approach: | They propose a matrix decomposition method that enables dynamic rank resource allocation . they say it can outperform current SOTA methods under various parameter budget levels . |
| Outcome: | The proposed method outperforms current SOTA methods under various budget levels . the proposed method is more efficient with higher compression rates . |